test(agent): regression coverage for #2169 skill load tool on fresh sessions#2369
test(agent): regression coverage for #2169 skill load tool on fresh sessions#2369gxgeek-n wants to merge 1 commit into
Conversation
|
|
There was a problem hiding this comment.
Pull request overview
Adds end-to-end regression coverage for #2169 by exercising a full ReAct loop with a real SkillBox + ReActAgent, asserting that load_skill_through_path remains authorized on fresh sessions and across repeated calls.
Changes:
- Introduces
SkillLoadToolFreshSessionTestwith scripted-model e2e tests to ensure the skill load tool executes (notUnauthorized) on the first call of a fresh session. - Verifies
ToolContextStateretains the build-time activeskill-build-in-toolsgroup after the call. - Verifies authorization persists across a second call within the same session.
| // Registered skill ids carry a "_custom" source suffix (see RegisteredSkill); | ||
| // the load tool's skillId enum only accepts the registered form. | ||
| String registeredSkillId = SKILL_ID + "_custom"; | ||
| Map<String, Object> input = new HashMap<>(); | ||
| input.put("skillId", registeredSkillId); | ||
| input.put("path", "SKILL.md"); |
…ol on fresh sessions Add end-to-end tests pinning the fix for agentscope-java#2169: a fresh-session ReActAgent built with a SkillBox must be able to execute load_skill_through_path instead of receiving "Unauthorized tool call: 'load_skill_through_path' is not available". The tests drive a scripted model through a real ReAct loop with a registered skill and assert on the tool result exactly as the model observes it: - fresh session: the load tool executes and returns the skill content - fresh session: ToolContextState keeps skill-build-in-tools activated - second call in the same session: the load tool stays authorized Verified both directions: all three tests fail on the v2.0.0 tag with the exact "Unauthorized tool call" error from the issue, and pass on main with the fresh-slot group seeding from agentscope-ai#2319.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
3ceaf28 to
0227ef5
Compare
|
CLA Not Signed The Contributor License Agreement (CLA) check is currently pending on this PR ( @gxgeek-n please sign the CLA via the CLA assistant badge in the comment above, or visit https://cla-assistant.io/agentscope-ai/agentscope-java. Once signed, the Automated check by github-manager-bot |
oss-maintainer
left a comment
oss-maintainer
left a comment
There was a problem hiding this comment.
LGTM. Looks good.
Automated review by "github-manager-bot"
Summary
Adds end-to-end regression tests pinning the fix for #2169 (
Unauthorized tool call: 'load_skill_through_path' is not availableon fresh sessions).The bug: on the first call of a fresh session,
activateSlotForContextapplied the state's (empty) activated group list viasetActiveGroups, deactivating the build-time activeskill-build-in-toolsgroup.SkillHookkept advertising skills in the system prompt, so the model inevitably calledload_skill_through_pathand got the Unauthorized error. This was fixed on main by #2319 (fresh slots inherit build-time active groups), but the end-to-end path through a realSkillBox+ReActAgentcall had no coverage — existing tests only assert state seeding with manually-created groups, andSkillBoxTest/SkillBoxToolsTestnever drive a full agent call.What the tests do
SkillLoadToolFreshSessionTestdrives a scripted model through a real ReAct loop with a registered skill, asserting on the tool result exactly as the model observes it:freshSession_skillLoadToolExecutes— the load tool executes and returns the skill content (not Unauthorized)freshSession_stateSeedsSkillGroup—ToolContextStatekeepsskill-build-in-toolsactivated after the callsecondCall_skillLoadToolStillAuthorized— activation survives repeated calls in one session (the wipe happened per call, not just on the first)Verified both directions
Error: Unauthorized tool call: 'load_skill_through_path' is not availablefrom the issue reportNotes
PreCallEvent; happy to drop it once a release ships with fix: preserve skill isolation and tool result history #2319